How important are specialized transforms in Neural Operators?

Simulating physical systems using Partial Differential Equations (PDEs) has become an indispensible part of modern industrial process optimization. Traditionally, numerical solvers have been used to solve the associated PDEs, however recently Transform-based Neural Operators such as the Fourier Neural Operator and Wavelet Neural Operator have received a lot of attention for their potential to provide fast solutions for systems of PDEs. In this work, we investigate the importance of the transform layers to the reported success of transform based neural operators. In particular, we record the cost in terms of performance, if all the transform layers are replaced by learnable linear layers. Surprisingly, we observe that linear layers suffice to provide performance comparable to the best-known transform-based layers and seem to do so with a compute time advantage as well. We believe that this observation can have significant implications for future work on Neural Operators, and might point to other sources of efficiencies for these architectures.Comment: 8 pages, 3 figures, 4 table

HyperLoRA for PDEs

Physics-informed neural networks (PINNs) have been widely used to develop neural surrogates for solutions of Partial Differential Equations. A drawback of PINNs is that they have to be retrained with every change in initial-boundary conditions and PDE coefficients. The Hypernetwork, a model-based meta learning technique, takes in a parameterized task embedding as input and predicts the weights of PINN as output. Predicting weights of a neural network however, is a high-dimensional regression problem, and hypernetworks perform sub-optimally while predicting parameters for large base networks. To circumvent this issue, we use a low ranked adaptation (LoRA) formulation to decompose every layer of the base network into low-ranked tensors and use hypernetworks to predict the low-ranked tensors. Despite the reduced dimensionality of the resulting weight-regression problem, LoRA-based Hypernetworks violate the underlying physics of the given task. We demonstrate that the generalization capabilities of LoRA-based hypernetworks drastically improve when trained with an additional physics-informed loss component (HyperPINN) to satisfy the governing differential equations. We observe that LoRA-based HyperPINN training allows us to learn fast solutions for parameterized PDEs like Burger's equation and Navier Stokes: Kovasznay flow, while having an 8x reduction in prediction parameters on average without compromising on accuracy when compared to all other baselines.Comment: 8 pages, 4 figures, 3 Table

Rate-Constrained Adaptive FEC for Video over Erasure Channels with Memory

Current adaptive FEC schemes used for video streaming applications alter the redundancy in a block of message packets to adapt to varying channel conditions. However, for many popular streaming applications, both the sourcerate and the available bandwidth are constrained. In this paper, we present FEC codes that can adapt in real-time to provide higher source-packets recovery without changing the FEC block (N, K) pair constraint. The FEC code profile is changed as function of the number of losses to facilitate an improved data recovery even under severe channel conditions (e.g., number of losses within an N- packet FEC block is larger than N-K). We present a feedback based adaptive FEC scheme, which can adapt in a rate-constrained manner. We also illustrate the utility of this scheme for video streaming applications by analyzing the results of extensive video simulations and comparing our performance to adaptive Reed Solomon FEC schemes. We consider a variety of video sequences and use actual packet traces from WLAN (802.11b) and wired Internet environments. Comparison between the two schemes is conducted on the basis of message packet recovery, PSNR, model based perceptual evaluation and visual subjective evaluation. It is shown that the proposed scheme can significantly improve the video quality and in particular reduce the jerkiness in the received video. 1

Scalable Video Transcaling for the Wireless Internet

The rapid and unprecedented increase in the heterogeneity of multimedia networks and devices emphasizes the need for scalable and adaptive video solutions both for coding and transmission purposes. However, in general, there is an inherent trade-off between the level of scalability and the quality of scalable video streams. In other words, the higher the bandwidth variation, the lower the overall video quality of the scalable stream that is needed to support the desired bandwidth range. In this paper, we introduce the notion of wireless video transcaling (TS), which is a generalization of (nonscalable) transcoding. With TS, a scalable video stream, that covers a given bandwidth range, is mapped into one or more scalable video streams covering different bandwidth ranges. Our proposed TS framework exploits the fact that the level of heterogeneity changes at different points of the video distribution tree over wireless and mobile Internet networks. This provides the opportunity to improve the video quality by performing the appropriate TS process. We argue that an Internet/wireless network gateway represents a good candidate for performing TS. Moreover, we describe hierarchical TS (HTS), which provides a &#147;Transcaler&#148; with the option of choosing among different levels of TS processes with different complexities. We illustrate the benefits of TS by considering the recently developed MPEG-4 fine granularity scalability (FGS) video coding. Extensive simulation results of video TS over bit rate ranges supported by emerging wireless LANs are presented.</p

ANALYSIS AND MODELING OF ERRORS AT THE 802.11b LINK LAYER

In this paper, we analyze the errors observed at the link layer of an 802.11b network. Our analysis at all supported bitrates (i.e., 2, 5.5. and 11 Mbps) establishes that the error patterns are not memoryless, and therefore, they exhibit a certain level of temporal dependencies. Thus, we evaluate the suitability of a two-state Markov model to capture the channel behavior. Non-stationarity of the error patterns renders such a simplistic model inadequate, and hence, we consider higher order models. This formulates a key contribution of this paper, and that is, a hierarchical Markov model, which captures the non-stationarity of the channel while employing real-time application-specific considerations to determine state-transition probabilities

Cross-Layer Protocol Design For Real-Time Multimedia Applications Over 802.11b Networks

Inherent vulnerability of the wireless medium renders it more susceptible to errors and losses than classical wired media. In this paper, we evaluate the suitability of protocols and strategies across different layers of the stack to provide real-time services over 802.11b wireless LANs. More specifically, within the context of cross-layer design, we compare the performance of UDP with UDP Lite - a proposed framework, which improves bandwidth utilization by delivering partially damaged packets to the realtime application. First, we study the high-level end-to-end throughput improvement achieved by making cross-layer modifications to support a UDP Lite framework. We compare the quality of perceived media rendered by UDP (dropped packets only) and UDP Lite (dropped and corrupted packets). This formulates one of the key findings of this study, that is, although UDP Lite improves the overall high-level throughput by relaying corrupted packets to the real-time application, it fails to provide significant enhancement in perceived media quality. This can, in part, be attributed to the bursty nature of errors and losses that we observed at the application layer regardless of the selected transport protocol. Finally, we compare the errorrecovery /concealment overhead required by UDP and UDP Lite in order to deliver lossless multimedia. We conclude that the overhead required by UDP Lite is considerably lower than UDP, since the received corrupted packets that are delivered by UDP Lite (but not by UDP) facilitate error-recovery